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Serial Number: 09/636,000 Dkt: BU2446/0033-033001 

Filing Date: August 9, 2000 

Title: Maximum Likelihood Sequence Estimator which Computes Branch Metrics in Real Time 

IN THE CLAIMS 

Please amend the claims as follows: 

1 . (Currently Amended) A Maximu m maximum likelihood sequence estimator 
(MLSE) for estimating a possible sequence of transmitted symbols received over a dispersive 
communication channel, wherein a trellis of states and trellis paths are associated with the 
possible sequence of transmitted symbols, said MLSE comprising: 

a plurality of data sources relating respectively to state transition probabilities and 
observed values of received data symbols; 

means for calculating and storing a likelihood metric and a survivor bit for each state of 
the trellis using values from said plurality of data sources, comprising means for computing 
branch metrics for the trellis by setting a first portion of the branch metrics equal to a coefficient 
which models an autocorrelation of an impulse response of a channel and setting a second 
portion of the branch metrics equal to a sum or difference of a prior value of a branch metric and 
a coefficient which models an autocorrelation of an impulse response of a channel; 

means for determination of a final state on a maximum likelihood path in the trellis; and 

means for calculating a maximum likelihood sequence of transmitted symbols in a 
backward trace through the trellis using said stored survivor bits. 

2. (Previously Presented) The MLSE of claim 1 further comprising means for 
computing supporting branch metric parameters, wherein said branch metric parameters are 
computed recursively for a Gray coded sequence of states, wherein said recursive computation 
requires only a single addition operation per branch metric parameter per state, thereby 
substantially reducing a number of computational steps required per branch metric parameter 
calculation. 

3. (Previously Presented) The MLSE of Claim 2 wherein said branch metric 
parameters are pre-computed and stored in a data memory prior to a forward trace through the 
trellis, wherein said stored branch metric parameters are retrieved from said data memory as 
needed to support state metric calculations subsequently performed in said forward trace through 
the trellis. 
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4. (Previously Presented) The MLSE of Claim 2 wherein said branch metric 
parameters are computed in real time as needed for state metric calculations, and wherein 
sequencing of said states is performed according to a Gray code for both branch metric 
calculations and state metric calculations, thereby achieving a substantial savings in a data 
storage requirement. 

5. (Previously Presented) The MLSE of Claim 1 further including means to utilize 
prior knowledge about the initial trellis state to enhance MLSE estimation performance of said 
transmitted sequence, said performance enhancing means including: 

an initial state register paired with an initial state mask register, wherein said pair of 
registers define a set of valid initial states representing prior knowledge about the transmitted 
sequence; and 

means for initialization of trellis state metrics such that an MLSE Viterbi algorithm 
selection of the maximum likelihood path in the trellis is confined only to paths having a valid 
initial state. 

6. (Currently Amended) A Maximum maximum likelihood sequence estimator 
(MLSE) for estimating a possible sequence of transmitted symbols received over a dispersive 
communication channel, wherein a trellis of states and trellis paths are associated with the 
possible sequence of transmitted symbols, said MLSE comprising: 

a plurality of data sources relating respectively to state transition probabilities and 
observed values of received data symbols; 

means for calculating and storing a likelihood metric and a survivor bit for each state of 
the trellis using values from said data sources; 

means for determination of a final state on a maximum likelihood path in the trellis; 

means for calculating a maximum likelihood sequence of transmitted symbols in a 
backward trace through the trellis using said stored survivor bits; and 

means to utilize prior knowledge about an initial trellis state to enhance MLSE estimation 
performance of said transmitted sequence, said performance enhancing means including: 
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a final state register paired with a final state mask register, wherein said register 
pair define a set of valid final states representing prior knowledge about the transmitted 
sequence; and 

means for selection of a final state of the trellis on the maximum likelihood path 
such that an MLSE Viterbi algorithm selection of the maximum likelihood path in the 
trellis is confined only to paths having a valid final state. 

7. (Previously Presented) The MLSE of Claim 2 further including means to provide 
sufficient data for a class of soft decision generators that are dependent only on partial path 
metrics. 

8. (Currently Amended) A method of computing a maximum likelihood sequence 
estimate comprising: 

providing an initial state; 

providing an initial state mask comprising a plurality of bits having either a first polarity 
or a second polarity; and 

determining a plurality of valid initial states by: 
starting with the initial state;-and 

substituting a don't care for each bit in the initial state which has a corresponding 
bit having a first state in the initial state mas k; and 

determining the maximum likelihood sequence estimate based on the plurality of 
valid initial states , 

wherein the valid initial states are defined by either a one or a zero in the bit 
position having a don't care, and the same bit as the initial state in the other positions. 

9. (Previously Presented) The method of claim 8 wherein the initial state mask is 
determined by power up characteristics of a transmitter. 



10. (Original) The method of claim 9 wherein the transmitter is compliant with the 
Global System for Mobile Communications standard. 
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1 1 . (Currently Amended) A method of computing a maximum likelihood sequence 
estimate comprising: 

providing a trellis comprising a plurality of nodes corresponding to a plurality of states at 
a plurality of stages;-and 

computing branch metrics for the trellis by setting a first portion of the branch metrics 
equal to a coefficient which models an autocorrelation of an impulse response of a channel and 
setting a second portion of the branch metrics equal to a sum or difference of a prior value of a 
branch metric and a coefficient which models an autocorrelation of an impulse response of a 
channel : and 

determining the maximum likelihood sequence estimate based on the branch metrics . 

12. (Previously Presented) The method of claim 1 1 wherein a present state is 
incremented to a next state by changing only one bit in accordance with a Gray code. 

13. (Previously Presented) The MLSE of Claim 6 comprising a comparator 
configured to test for equality of an output of a final state register and a current state in 
accordance with an output of the final state mask register. 

14. (Previously Presented) The MLSE of Claim 13 comprising a multiplexer for 
selecting the current state or a prior state in accordance with the test for equality. 

15. (Previously Presented) The MLSE of Claim 14 comprising a latch for storing the 
current state in accordance with the test for equality. 

16. (Previously Presented) The MLSE of Claim 6 comprising means to utilize prior 
knowledge about the initial trellis state to enhance MLSE estimation performance of said 
transmitted sequence, said performance enhancing means comprising: 

an initial state register paired with an initial state mask register, wherein said pair of 
registers define a set of valid initial states representing prior knowledge about the transmitted 
sequence; and 
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means for initialization of trellis state metrics such that the MLSE Viterbi algorithm 
selection of the maximum likelihood path in the trellis is confined only to paths having a valid 
initial state. 

17. (Previously Presented) The method of claim 8 comprising testing for equality of 
the initial state and a current state in accordance with the initial state mask. 

18. (Previously Presented) The method of claim 17 comprising controlling a 
multiplexer in accordance with the testing for equality of the initial state to select an initial 
metric value associated with a valid initial state or an initial metric value associated with an 
invalid initial state. 

19. (Previously Presented) The method of claim 8 comprising associating initial 
metric values with the valid initial states and invalid initial states, wherein the initial metric 
values associated with the valid initial states are greater in magnitude than the initial metric 
values associated with the invalid initial states. 

20. (Previously Presented) The method of claim 19 wherein the initial metric values 
associated with the invalid initial states are negative values. 

21 . (Previously Presented) The method of claim 8 comprising: 
providing final state data; 

providing a final state mask; 

determining a plurality of valid final states in accordance with the final state data and the 
final state mask; and 

selecting path matrices corresponding to a valid initial state and a valid final state. 



22-30. (Canceled) 



